1. Field of the Invention
The present invention relates generally to image pattern recognition, and more particularly to face image processing systems including detection and monitoring of the open or closed state of at least one eye image as extracted using face image processing techniques. The instant invention also relates to a face image processing apparatus for use in security systems for maintaining the continuous of workers safety by raising an alarm upon the occurrence of a doze or sleep while at work.
2. Description of the Prior Art
As one of image pattern recognition techniques, a face image processing system has a wide variety of applications, including an apparatus for detecting falling asleep of a target person, such as an automobile driver, by monitoring the open/closed state of his or her eyes. One of such driver's condition monitoring systems has been described, for example, in Published Unexamined Japanese Patent Application No. 6-227278, which is arranged generally as shown in FIG. 33. The prior art apparatus of FIG. 33 includes an image input section 82 for acquiring an input face image of a driver, a digitizer 83 for digitizing the face image acquired by the image input section 82 to produce a binary face image signal, and an eyeball presence region setter 56 for determining the exact location of an eyeball image region in the binary face image. An eye open/closed state detector 57 is adapted to detect the open or closed state of an eye within the eyeball presence region thus detected. The detector 57 is connected to a driving condition identifier 86 which identifies the actual condition of the driver on the basis of the resulting eye open/close pattern detected by the eye open/close detector 57. A detection error identifier 84 is provided for determining the occurrence of any error in detection of the eyeball presence region based on the detection results of the open/close detector 57. A resetter 85 is also provided for modifying or updating the eye presence region in response to the detection error identification results.
The operation of the prior art apparatus will be described with reference to FIGS. 33 through 36. When a face image of a target driver is acquired and inputted by the image input section 82, the resultant face image is then supplied to the digitizer 83, which digitizes it to produce a binary face image signal. This image signal is given to the eyeball presence region setter 56, which extracts an eyeball image portion from the face image to determine and set the location of such eyeball, i.e., the eyeball presence region therein. The open or closed state detection of the open/close detector 57 is effected within a window 58 (see FIG. 34) taken out of the face image so as to contain the eyeball image portion as set by the eyeball presence region setter 56. The open/close detector 57 may employ one of two different kinds of schemes as the open/closed state detecting method thereof as will be described below.
With a first eye open/closed state detection scheme, an iris is detected in the face image window 58 of FIG. 34 representing the eyeball presence region. As shown in FIG. 35, such iris may be detected by (i) defining a certain circle 59 defined with a given point (x, y) being as a center within the window 58 and four slit-like rectangular marks 60 that extend from the center point (x, y) in four radial directions, (ii) calculating any possible difference in the total sum of brightness values of respective white portions of the rectangular marks 60 and that of hatched portions thereof, and (iii) determining as the iris center a point that exhibits a maximal brightness difference.
The eye-open/closed state detection is effected based on detection of such maximal brightness difference at the iris center thus determined. At this time, if any maximal brightness difference is not detected for a predetermined time period, the detection error identifier 84 then determines that the eye detection fails and a detection error is occurred. Once such eye detection error takes place, the resetter 85 forces the eyeball presence region setter 56 to perform resetting of the eyeball presence region.
A second eye open/closed state detection scheme is as follows. As shown in FIG. 36, the eye window 58 is vertically scanned to search for any contiguous black colored picture elements (pixels) in order to detect an eyeball image, thereby to find out a specific image portion consisting of a maximum number 63 of contiguous black pixels. Calculating this maximum contiguous black-pixel number 63 is done repeatedly with respect to a plurality of images; the resultant maximum and minimum values of them are used to set a suitable threshold value. After the threshold value is set, the open/closed state of eye(s) may be detected by comparing the maximum contiguous black-pixel number 63 to the threshold value. At this time, the detection error identifier 84 may detect that the maximum contiguous black-pixel number 63 remains unchanged during a predetermined time period; if this is the case, it is concluded that no correct eye detection is made. Responding to the determination of such eyeball detection error, the resetter 85 then causes the eyeball presence region setter 56 to perform an eyeball presence region resetting operation, i.e., repeating a similar eyeball presence region setting operation with respect to a different face image region.
As is seen from the above explanation, the prior art apparatus is so arranged as to acquire necessary information used to determine whether a driver is falling asleep or not by analyzing variations of an iris portion which is detected from an eye window taken from a binary face image. Alternatively, the prior art apparatus is arranged to (i) detect a possible eyeball presence region, from which the information as to whether a driver is dozing is obtainable, by vertically scanning the eye window 58 to search for a series of contiguous black pixels therein, (ii) calculate the maximum contiguous black-pixel number, and (iii) finally determine whether the eyeball detection is successful or not by analyzing monitored variations in the maximum contiguous black-pixel number during a predetermined time period.
A drawback of the prior art is that the eyeball detection is unstable and less reliable for the following reasons. A driver's face image sensed may involve shadows due to variations in image pickup light rays incident on his or her face and/or variations in driver's face direction at the time of image pickup. If this is the case, the binary eyeball presence region extracted from a digitized face image may become unstable, causing an intended eyeball image to shift in position from a regular location or letting an eyeball image be deformed into an ellipsoidal shape. As far as detection of the open/closed state of an eye is performed by detecting an iris image under such an unstable condition, the prior art apparatus is incapable of suppressing or eliminating the occurrence of detection errors.
Another drawback of the prior art is that an erroneous threshold value may possibly be set depending upon the actual binary state of an input face image, causing eye-open/close detection to fail due to the fact that the prior art apparatus focuses attention only to an eye portion of the eyeball presence region in the input face image and determines a threshold value with such eye portion being as a reference.